Real-time display of system instructions

ABSTRACT

A system and method for reviewing inputted voice instructions in a vehicle-based telematics control unit. The system includes a microphone, a speech recognition processor, and an output device. The microphone receives voice instructions from a user. Coupled to the microphone is the speech recognition processor that generates a voice signal by performing speech recognition processing of the received voice instructions. The output device outputs the generated voice signal to the user. The system also includes a user interface for allowing the user to approve the outputted voice signal, and a communication component for wirelessly sending the generated voice signal to a server over a wireless network upon approval by the user.

PRIORITY CLAIM

The following application is incorporated by reference as if fully setforth herein: U.S. application Ser. No. 10/076,874 filed Feb. 14, 2002.

FIELD OF THE INVENTION

This invention relates generally to the field of vehicle userinterfaces.

BACKGROUND OF THE INVENTION

Various vehicle devices exist that are designed to provide a vehicleoperator with travel-related information. For example, a display coupledto a global positioning system (GPS) may be used to present to thevehicle operator a detailed map showing the vehicle location, a desireddestination, and the geographic details of the surrounding area. Undercertain situations, the vehicle operator has the opportunity to studythe map in exact and thorough detail. An example is a parked vehicle.There are other times, however, when the vehicle operator is too busyoperating the vehicle to safely view the displayed information, letalone study the displayed information in exacting detail. This isparticularly the case where the vehicle operator is in the midst of aparticularly taxing circumstance, for example, driving in heavy traffic,driving at a high speed, or otherwise distracted by an emergencysituation or telephone conversation. In such situations, the displaypresentation distracts the vehicle operator because the operator focusesattention on the details shown on the display.

Therefore, there exists a need to provide valuable and timelytravel-related information to a vehicle operator while minimizingpotentially dangerous distractions to the vehicle operator.

SUMMARY OF THE INVENTION

The present invention is a system and method for reviewing inputtedvoice instructions in a vehicle-based telematics control unit. Thesystem includes a microphone, a speech recognition processor, and anoutput device. The microphone receives voice instructions from a user.Coupled to the microphone is the speech recognition processor thatgenerates a voice signal by performing speech recognition processing ofthe received voice instructions. The output device outputs the generatedvoice signal to the user. The system also includes a user interface forallowing the user to approve the outputted voice signal, and acommunication component for wirelessly sending the generated voicesignal to a server over a wireless network upon approval by the user.

In accordance with further aspects of the invention, a first componentgenerates a digest including the generated voice signal, and a secondcomponent sends the digest to a human operator system.

In accordance with other aspects of the invention, a text generatorgenerates text from the generated voice signal, and the output deviceincludes a display that presents the generated text.

In accordance with still further aspects of the invention, a voicesynthesizer generates speech based on the generated voice signal, andthe output device includes at least one speaker for presenting thegenerated voice.

As will be readily appreciated from the foregoing summary, the inventionprovides a feedback system for ensuring accuracy when providing voiceinstructions.

BRIEF DESCRIPTION OF THE DRAWINGS

The preferred and alternative embodiments of the present invention aredescribed in detail below with reference to the following drawings.

FIG. 1 is a block diagram illustrating the components of the presentinvention;

FIG. 2 is a flow chart illustrating the present invention performed bythe device shown in FIG. 1;

FIG. 3 is a flow chart illustrating the operation of a specific exampleof the present invention;

FIG. 4 is a block diagram illustrating the components of an alternateembodiment of the present invention; and

FIG. 5 is a flow chart illustrating the present invention performed bythe device shown in FIG. 4.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

The present invention provides a system, method, and computer programproduct for modulating output over output devices (e.g., display,speakers) based on present vehicle conditions (e.g., speed, rate ofchange in speed or direction, emergencies). When the system determinesthat the vehicle requires a more attentive operator, the method of thepresent invention manipulates the format of information and contentpresented to the vehicle operator via the output devices.

FIG. 1 shows a system 20, a telematics control unit, within a vehiclethat performs the present invention. System 20 includes a user interface22 having a microphone 24 for capturing the user's voice; a display 26;speakers 28; and an interface entry 30 for allowing the user to performvarious interactive functions. System 20 also includes a processor 32; aglobal positioning system (GPS) 34 for determining precise vehiclelocations; and a communications device 36, such as a cellular modem, fortransmitting and receiving wireless information. System 20 is alsocoupled to receive information from other vehicle systems (not shown),such as a speedometer, an engine, or a braking system.

FIG. 2 illustrates a preferred process performed by system 20. At block60, system 20 and, more specifically, processor 32, receivespresentation content. The presentation content is preferably received atcommunications device 36 from a server transmitted over a network (notshown). At decision block 62, processor 32 determines if any vehicleemergency condition exists based on information received from the othervehicle systems. If processor 32 determines that a vehicle emergencycondition exists, processor 32 stops all output of received presentationcontent to any one of the components of user interface 22 (block 64).Examples of a vehicle emergency condition that may cause the processorto stop output presentation content to user interface 22 are signalsreceived from the engine that indicate engine overheating or a signalthat indicates the operator has applied brakes to the point where ananti-braking system is activated.

If, at decision block 62, processor 32 determines that there is not avehicle emergency condition, processor 32 receives vehicle conditioninformation (block 66). The vehicle condition information comes from anumber of sources that generate signals, such as GPS 34 or aspeedometer. At block 68, processor 32 outputs presentation contentthrough user interface 22 based on the vehicle condition information.This step involves determining the substance and format of thepresentation content to display on display 26 or broadcast on speakers28 of user interface 22. For example, when the vehicle is parked,processor 32 presents all or nearly all of the received presentationcontent. As the vehicle condition changes, for example, the vehiclepicks up speed, processor 32 changes the presentation contentinformation that is shown on display 26. The change in presentationcontent may also involve shifting presentation of information fromvisual display to audio broadcast as output through speakers 28. In oneexample, the presented content is a full detailed map of the vehicle'slocation that includes directions to a previously requested destination.Based on the vehicle speed, or the rate of change in vehicle speed ordirection, the detail shown on the map will be altered. As the vehiclespeed increases, the detail shown on the map is reduced, for example, toinclude only the vehicle's present location in relation to nearbystreets or intersections. Some or all of the presentation content, ormap directions, may be translated to audio format and broadcast to thevehicle operator via speakers 28. In other examples, change in vehiclecondition may result in alteration of the font or size of text, iconsand graphics presented on the display, as well as increased or decreasedaudio broadcast. FIG. 3 illustrates an example, based on vehicle speed,of how processor 32 changes the output that is presented over userinterface 22.

After blocks 64 and 68, processor 32 returns to block 60 to continuereceiving more presentation content. In this embodiment, presentationcontent is processed in real-time from the time of reception from theserver over the network to the output of presentation content to avehicle operator through user interface 22. In an alternate embodiment,the presentation content can be received all at one time and stored byprocessor 32 before any determination is made of how to present theoutput. In another alternate embodiment, if a person other than thevehicle operator wishes to view the presentation content, presentedoutput is set to whatever level of detail is desired.

FIG. 3 illustrates an example of the process performed at block 68 ofFIG. 2. At block 80, vehicle speed information is determined, forexample by reference to the vehicle speedometer. If the vehicle's speedis less than or equal to 25 mph, processor 32 generates a detaileddisplay to present on display 26 of user interface 22 (block 82). As thespeed of the vehicle increases, the amount of detailed visualinformation generated for display is reduced. Thus, if the vehicle speedis greater than 25 mph but less than or equal to 40 mph, processor 32generates a display with less visual information than was generated forvehicle speeds of less than or equal to 25 mph (block 84). When thespeed is greater than 40 mph but less than or equal to 70 mph, processor32 reduces not only the amount of detail to be displayed on display 26,but also increases the font size of any text included in the displayedpresentation content and changes the color to make it easier for anoperator to view (block 86). When the speed is greater than 70 mph,processor 32 stops generating display information for presentation ondisplay 26 and switches to a pure audio presentation for output over thespeakers 28 (block 88). The specific speed ranges described areexemplary only. The exact trigger speeds may vary according to variousfactors, such as road conditions, vehicle specification, vehicleoperator skill, etc., as well as user preference. Likewise, the outputformat of the presentation content may be varied according to userinterface hardware or software constraints as well as user preference.

In addition to the example described above with reference to overallvehicle speed, presentation content can be automatically adjusted basedupon one or more vehicle conditions related to rate of change in vehiclespeed, rate of change in vehicle direction, cellular phone or headphoneactivation, radar detector activation, noise level decibels above acertain value, radio/CD player in use, TV/DVD console in use, turnsignal, etc. The vehicle conditions are determined by vehicle systems,such as the vehicle speedometer, steering, engine, and braking andacceleration systems. Vehicle systems may also include additionalcomponents, such as temperature, humidity, wind speed and precipitationgauges, a compass, clock, etc. These additional vehicle systemcomponents may be integrated with or separate from system 20, so long asthey are capable of being monitored by system 20. The method of thepresent invention automatically adjusts the output of presentationcontent based upon a monitor of these vehicle conditions, or acombination of these vehicle conditions. For example, system 20 monitorsvehicle braking and acceleration systems with reference to an internalclock to determine the rate of change of vehicle speed. If vehicle speedis increasing or decreasing rapidly, system 20 automatically reduces theamount of detail provided in visual presentation content, and maytransfer some or all of the content to audio presentation. In anotherexample, system 20 monitors movement of the steering wheel and vehicledirection with reference to a compass and a clock to determine rate ofchange of vehicle direction. If vehicle direction is changing rapidly,system 20 would again automatically reduce the amount of detail providedin visual presentation content, as well as transfer some or all of thecontent to audio presentation.

FIG. 4 illustrates a vehicle system 120, a telematics control unit, thatperforms an alternate embodiment of the present invention. Vehiclesystem 120 includes a user interface 122 and a processor 124. Userinterface 122 includes a microphone that receives voice instructionsfrom a user and sends them to processor 124. Processor 124 performsvoice processing of the received voice signals. The processor outputsthe processed voice signals over speakers or a display that are part ofuser interface 122. Also, processor 124 sends the processed voicesignals to an operator 142 at an operator system 140 over a wirelessnetwork 130.

FIG. 5 illustrates a process performed by system 120 shown in FIG. 4.First, at block 210, the user inputs voice instructions to system 120(processor 124) via user interface 122. Next, at block 212, processor124 performs voice recognition processing on the received voiceinstructions to determine the content of the inputted instructions. Theresult of the voice recognition processing is presented to the user viauser interface 122 in the form of voice over speakers or text on adisplay (block 214). If processor 124 has interpreted the inputted voiceinstructions properly, as shown in decision block 216, the userinstructs system 120 to send the processed instructions to a predefineddestination (block 218). The predefined destination is preferably asystem server. Otherwise, the user has the opportunity to contact ahuman operator to make sure the instructions are properly entered(decision block 220). If the user decides to contact the human operator,a digest of the interaction between the user and processor 124 is sentto human operator system 140 (block 222). Then, at block 224, the useris connected to the human operator who is tending a phone and/orcomputer terminal. At this point the user and human operator resolve thecontent of the user-initiated instructions. If at the decision block 220the user does not approve of the presented voice instructions or desireto be connected to a human operator, the user can return to providingnew voice instructions (block 210).

In an alternate embodiment, processor 124 processes a received voiceinstruction based on previously recorded prior processing of the samevoice instruction. For example, at five previous episodes, the userprovided a certain voice instruction about a restaurant. Processor 124made a first determination as to what the user requested for each of thefive episodes. In the first four of the five episodes, the user desireddirections to the restaurant. In the fifth episode, the user desiredstock information about the publicly traded company by the same namethat owns the restaurant. The approved results (four directions; onestock information) of the processed voice instruction are recorded atprocessor 124 with the voice instruction for each episode. The next timethe user provides that same voice instruction, processor 124 uses thepreviously recorded approved results when processing the voiceinstruction. Various methods of using the previously recorded approvedresults can be performed. In a first method, the most frequent recordedresult becomes the result of the newly provided specific voiceinstruction. So, using the example above, directions are provided to theuser, because the recorded results show requests for directions to therestaurant (as opposed to requests for stock information) occurring morefrequently. In a second method, the most recent recorded result becomesthe result of the newly provided specific voice instruction. So, usingthe example above, stock information is provided to the user, becausethe last recorded result was stock information.

With respect to the first four episodes described above, processor 124processed the voice instruction as either a request for directions or asa request for something other than directions. If the voice instructionwas processed as a request for directions, the user approves of theprocessing, and the resulting request for directions to the restaurantis recorded with the voice instruction for the episode. If the voiceinstruction was processed as a request for something other thandirections, the user makes a request that processor 124 providedirections, thereby indicating that the processing of the voiceinstructions was incorrect. The user can indicate that the processingwas incorrect (and in this example that directions were desired) in anumber of different ways. One way to make a change request is to contacta human operator (decision block 220 of FIG. 5). The human operatormakes the change for that episode, and the change is recorded. Anotherway is to make a subsequent voice instruction requesting processor 124to reinterpret the original voice instruction. Once the voiceinstruction has changed to a request for directions and the userapproves of the change, the result of directions to the restaurant isrecorded with the voice instruction for that episode.

While the preferred embodiment of the invention has been illustrated anddescribed, as noted above, many changes can be made without departingfrom the spirit and scope of the invention. For example, as noted above,the trigger event or threshold used to determine the output format ofpresentation content, as well as the specific output format, might varyaccording to different factors as well as user preference. Accordingly,the scope of the invention is not limited by the disclosure of thepreferred embodiment. Instead, the invention should be determinedentirely by reference to the claims that follow.

1. A method for reviewing inputted voice instructions in a vehicle-basedtelematics control unit, the method comprising: recording voiceinstructions from a user; generating a voice signal by performing speechrecognition of the recorded voice instructions; and outputting thegenerated voice signal over an output device associated with thetelematics control unit for review.
 2. The method of claim 1, furthercomprising wirelessly sending at least one of the generated voice signalor the inputted voice instructions to a server over a wireless networkupon approval by a user.
 3. The method of claim 1, further comprising:generating a digest including the generated voice signals; sending thedigest to a human operator system; and connecting the human operatorsystem to the telematics control unit.
 4. The method of claim 1, whereinoutputting comprises generating and displaying text based on thegenerated voice signal.
 5. The method of claim 1, wherein outputtingcomprises generating and outputting voice based on the generated voicesignal.
 6. A system for reviewing inputted voice instructions in avehicle-based telematics control unit, the system comprising: amicrophone for receiving voice instructions from a user; a speechrecognition processor coupled to the microphone for generating a voicesignal by performing speech recognition processing of the received voiceinstructions; and an output device for outputting the generated voicesignal to the user.
 7. The system of claim 6, further comprising: a userinterface for allowing the user to approve of the outputted voicesignal; and a communication component for wirelessly sending thegenerated voice signal to a server over a wireless network upon approvalby the user.
 8. The system of claim 6, further comprising: a firstcomponent for generating a digest including the generated voice signal;and a second component for sending the digest to a human operatorsystem.
 9. The system of claim 6, further comprising a text generatorfor generating text from the generated voice signal, wherein the outputdevice includes a display for presenting the generated text.
 10. Thesystem of claim 6, further comprising a voice synthesizer for generatingvoice based on the generated voice signal, wherein the output deviceincludes at least one speaker for presenting the generated voice.
 11. Amethod for predicting processing of a voice instruction; the methodcomprising: providing a voice instruction a first time; processing thevoice instruction; generating a result based on the processed voiceinstruction; approving the generated result; recording the approvedresult in association with the provided voice instruction; providing thevoice instruction at a second time; and processing the voice instructionthat was provided the second time based on the recorded result of thevoice instruction that was provided at the first time.
 12. The method ofclaim 11, wherein approving the generated result comprises correctingthe generated result.
 13. The method of claim 12, wherein correctingcomprises contacting an operator.
 14. The method of claim 11, whereinapproving the generated result comprises selecting from a list ofchoices.